Analysis of Asymmetric Measures for Performance Estimation of a Sentiment Classifier

The development of a sentiment classifier experiences two problems to cope with: the demand of large amounts of labelled training data and a decrease in performance when the classifier is applied to a different domain. In this paper, we attempt to address this problem by exploring a number of metrics that try to predict the cross-domain performance of a sentiment classifier through the analysis of divergence between several probability distributions. In particular, we apply similarity measures to compare different domains and investigate the implications of using non-symmetric measures for contrasting feature distributions. We find that quantifying the difference between domains is useful to predict which domain has a feature distribution most similar to the target domain.